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A large gap exists at present between computational theories of vision and their possible 
implementation in neural hardware. The model of computation provided by the digital 
^^^^ computer is clearly unsatisfactory for the neurobiologist, given the increasing evidence that 

f^ neurons are complex devices, very different from simple digital switches. It is especially 

difficult to imagine how networks of neurons may solve the equations involved in vision 
algorithms in a way similar to digital computers. In this paper, we suggest an analog model 
of computation in electrical or chemical networks for a large class of vision problems, that 
maps more easily into biologically plausible mechanisms. Poggio and Torre (1984) have 
recently recognized that early vision problems such as motion analysis (Horn and Schunck, 
1981; Hildreth, 1984a, b), edge detection (Torre and Poggio, 1984), surface interpolation 
(Crimson, 1981; Terzopoulos 1984), shape-from-shading (Ikeuchi and Horn, 1981) and 
stereomatching can be characterized as mathematically ill-posed problems in the sense of 
Hadamard (1923). Ill-posed problems can be "solved", according to regularization theories, 
by variational principles of a specific type. A natural way of implementing variational 
problems are electrical, chemical or neuronal networks. We present specific networks for 
solving several low-level vision problems, such as the computation of visual motion and 
edge detection. 
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Ill-Posed Problems and Variational Principles 

In mathematical analysis ill posed problems are defined as such problems that do not have a 
unique solution that depends continuously on the data. Early vision problems are ill-posed, 
typically because the solution is either not uniquely constrained by, or does not depend 
continuously on the data. Computation of the velocity field and the recovery of shape 
from shading are examples of the first type; edge detection, considered as the problem 
of measuring spatial changes of intensity in an image, is an example of the second class. 
Rigoreous regularization theories for "solving" ill-posed problems have been developed 
in recent years (see especially Tikhonov and Arsenin, 1977; and Nashed, 1974). Ill-posed 
problems of the form 

y = Az, (i) 

where A is a possibly nonlinear operator on the unknown solution z. and y is the data, 
can be regularized by introducing suitable norms ||-||, usually quadratic, and a "stabilizing" 
functional \\Pz\\, with certain mathematical properties. Its exact form depends on the 
physical constraints that are relevant to the specific problem. The regularized solution is 
then obtained in terms of a variational principle. For simplicity, we consider here two main 
methods (the second and third methods in Poggio and Torre, 1984): 

(a) If the data are exact, then the regularized solution z is the function that minimizes 

II Nl (2) 

among the z such that y = Az. 
r ' (b) If the data are not exact, the regularized solution is z that minimizes 

||/l*-l,|| s + X||P*||» (3) 

where X is a regularizing parameter (see Poggio and Torre, 1984). 

Typically, A and P are linear operators and P usually corresponds to a physically relevant 
smoothness constraint. Poggio and Torre (1984) have argued that most early vision problems 
can be solved in terms of variational formulations of this type. A few such solutions have 
been already obtained. A representative example is the computation of the velocity field 
(Hildreth, 1984a,b; see also Poggio and Torre, 1984; for a second example see Poggio, 
Voorhees and Yuille, 1984) 1 . In this case the velocity V of each point along a curve in the 
image is underdetermined, since 

V(a) = v T {s)l{a) + t»-L(a)N(a) (4) 

(T, N are the unit vector tangent and perpendicular to the contour respectively, a is the 
arc length), and only v T (a) can be obtained by local measurements, if the curve is smooth. 
The stabilizing functional derived by physical considerations (Hildreth, 1984a,b) is obtained 
from P\l = D\f/ds. The regularized solution is found by 

(a) minimizing 
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subject to the data V T , if the data are exact, or by 
(b) minimizing 



||7 T -V-T|| 2 + X||^|| 2 , (6) 

da 

where X -1 is a weighting factor expressing the reliability of the velocity measurements 2 , 
when the data are affected by errors. 



Analog Networks for Solving Variational Problems 

As suggested by Terzopoulos (1984), analog networks— chemical, electrical or mechanical- 
are a natural computational model for solving variational principles. We know from physics 
that the behavior of such systems, in fact the behavior of any physical system, can 
be described using variational principles (MacFarlane, 1970). In the frictionless world of 
classical mechanics a system's state variables will behave in such a way as to minimize 
the associated Lagrangian. Electrical network representations have been constructed for 
practically all of the field equations of physics — many of them are equivalent to variational 
principles (for an electrical network implementation of Schredinger's equation see Kron, 
1945). A fundamental reason for a natural mapping between variational principles and 
electrical or chemical networks is Hamilton's Least Action principle 3 . The class of variational 
principles that can be computed by analog networks is dictated by Kirchhoff's current and 
voltage laws (KCL and KVL), which simply represent conservation and continuity restrictions 
satisfied by each network component (appropriate variables are usually voltage and current 
for electrical networks and affinity, i.e. chemical potential, and chemical turnover rate for 
chemical systems). KCL and KVL provide the unifying structure of network theory, A large 
body of theoretical results is available about networks satisfying them, including classical 
/"*\ thermodynamics (Oster, Perelson and Katchalsky, 1971). In particular, KCL and KVL imply 

Tellegen's theorem 1 . For a network containing only sources and linear resistors, Tellegen's 
theorem implies Maxwell's minimum heat theorem: the distribution of voltages and currents 
is such that is minimizes the total power dissipated as ftear 5 . These results can be extended 
to nonlinear circuit components (MacFarlane, 1970; Oster & Desoer, 1971), but in the 
following we will restrict ourselves to linear resistors. Since the power dissipated as heat 
by linear resistors is' a quadratic functional of the form 



f~\ 



X>n 



(7) 



where h and V k are the current and the voltage respectively, and the sum is taken over all 
resistive processes r k , it follows that any network consisting of linear resistors and voltage 
sources E k has the following associated quadratic functional that is minimized 

5>/5-2>j, (8) 

k i 

where the second sum includes all the batteries. The functional for the network containing 
current sources is very similar 7 . 

It is then easy to show formally that electrical networks of linear resistors and batteries 
(or current sources) can solve quadratic minimization problems in a Hilbert space 8 . The 
corresponding linear equations Qz = b have a unique solution z, corresponding to the unique 
solution of the variational principle. Interestingly, the data 6 can always be represented 
in terms of current or voltage sources. The matrix Q is the symmetric, real matrix of the 
network resistances 8 . 

A simpler argument also shows that variational principles of the type of equation (2) 
and (3) can be solved by linear networks. The Euler-Lagrange equations associated with 



equations (2) and (3) are linear in z. In the discrete case, they correspond to n linear, 
coupled algebraic equations that can be solved by networks of passive, linear resistors and 
voltage/current sources". An alternative implementation, common on analog computers, 
involves operational amplifiers (Jackson, 1960). As pointed out by Terzopoulos in the 
context of vision (earlier, Horn (1974) proposed an analog implementation of the lightness 
computation), a significant advantage of an analog implementation is its extreme parallelism 
and speed of convergence. Another advantage of resistance networks is their robustness 
against random errors in the resistance units of the network (Terzopoulos, 1984; Karplus, 
1958). A disadvantage is the limited precision of the analog signals. 






Figure 1: Three resistive networks computing the smoothest velocity field solution of Hildreth, 1984a. 
The first two networks correspond to the situation where the constraints imposed by the data are to 
be satisfied exactly. The equation for the current, which corresponds to the desired v T in mesh i 
(for figure 1a), is given by (2r + r,)/< - r] i+i - r/,_i = E it where the value of the battery E> depends 
on the velocity data v x at location t. The voltage at node i, corresponding to vj, for the network 
1b, the dual of network 1a, is given by {2g + </,)K,- - gV i+i - sVi_i = /,-, where the injected current /,• 
depends on the velocity data. Sampling the voltage between nodes corresponds to linear interpolation 
between the node values. Network 1c, consisting of two interconnected networks of the type shown 
in 1b, solves the velocity field problem when the data are not exact. The equations for the i-th nodes 
are (2g x + g xi )V xi -g z V x i +1 - g x V xi -i + c,V v ,- = d xi and {'2g v + g V i)V vi - f?„V„, + i - <7 v V v ,_i + c,V xj = <£„,-. 
However, unlike the two purely passive networks shown above, an active element may be required, 
since the cross-term c,-, relating the x and the y components of velocity, can be negative. Such a 
negative resistance can be mimicked by operational amplifiers. 



An Example: Circuits for the Velocity Field Computation 

We will consider next some specific networks for solving the optical flow computation 10 . 
The simpler case is when the measurements of the perpendicular component of the velocity, 
v±, at n points along the contours are exact. In this case, the discretized Euler-Lagrange 
equations, corresponding to the regularization solution, equation (5), are (Hildreth, 1984a) 



(2 + KfivJ - «jVi 



T 



„T 



v JLi = di, 



(9) 



where k is the curvature of the curve at location i, d t is a function of the data v^- and the 
curve and vj is the unknown tangential component of the velocity «,- at location i to be 
computed. Figure 1a and 1b shows two simple networks that solve equation (9), where one 
network is the dual of the other. 
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Figure 2: Two examples of chemical networks solving the motion problem for exact measurements. 
They are equivalent, under steady-state conditions, to the electric circuit of figure 1b. Fig. 2a 
illustrates a diffusion-reaction system. A substance A (the concentration of which corresponds to 
the desired v T ) diffuses along a cable while reacting with an extracellular substance S (first order 
kinetics). The corresponding On-rate k t varies from location to location. This could be achieved by 
a differential concentration of an enzyme catalysing the reaction or by varying the properties of 
the membrane where the reaction has to take place. The Off-rates can be either constant or vary 
with location. The inputs are given by the influxes of substance A. Figure 2b shows an instance of 
a lumped chemical network, n different, well-mixed substances, interact with each other and with 
the substrate S. Assuming first-order kinetics, these reactions can mimick a linear resistance under 
steady-state conditions. The input is given by the influx Mx and the output by the concentration- of 
X. Chemical networks of this type are a kind of computational soup. 



The equation describing the i-th node, in the case of figure 1b, is 



f~\ [2g + gi )Vi - gV i+l - gV^ = /,• (10) 

where V { is the voltage— corresponding to the unknown v T — and 7; the injected current 
at node {—corresponding to the measurements ?<- L . It is important and surprising that this 
implementation does not require negative resistances. When the constraints are satisfied 
only approximately (equation (6)), the equations are 



(2 + b 2 xi )V Xi - K Xj+1 - V S| _, + aV yi = d 
^ + bl)V y ,-V y , +1 -V yi _ 1 +c i V x% =d 
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where fc,- depends on the contour and V x , and V yi denote the x and y component of the 
unknown velocity «, at location i. The corresponding network is shown in Figure 1c. The 
resistances c,-, however, can be either positive or negative, and may therefore require active 
components such as operational amplifiers 11 . In the limit, as the meshes of the circuit 
become infinitesimally small, the network solves the continuous variational problem, and 
not simply its discrete approximation. 

We have devised similar analog networks for solving other variational problems arising from 
regularization analysis of several early vision problems such as edge detection (Poggio 
et al., 1984) and surface interpolation 12 (Terzopoulos, 1984). These networks are analog 
solutions to certain kinds of spline interpolation and approximation problems. For instance, 
in the case of surface interpolation the analog network solves the biharmonic equation 
which is the Euler-Lagrange equation corresponding to the variational problem associated 
with thin-plate splines. The stabilizing Junctionals used in regularization analysis of vision 
problems typically lead to local and limited connections between the components of the 
network. 



Solving Ill-Posed Problems with Biological Hardware 

The mathematical property of ill-posedness determines a common computational structure 
for the solution to specific problems in early vision based on variational principles of 
a specific type. Analog electrical networks are a natural hardware for computing such 
variational principles. Because of the well-known isomorphism between electrical and 
chemical networks (see for instance Busse and Hess, 1973 or Eigen, 1974) that derives from 
the common underlying mathematical structure, appropriate sets of chemical reactions can 
be devised, at least in principle, to "simulate" exactly the electrical circuits. Fig. 2 shows 
two chemical networks that are equivalent (in the steady state) to the electrical circuit of 
fig. 1b 13 . 

Electrical and chemical systems of this type therefore offer a computational model for early 
vision that is quite different from the digital computer. Equations are "solved" in an implicit 
way, exploiting the physical constraints provided by Kirchhoff's laws. It is not difficult to 
imagine how this model of computation could be extended to mixed electrochemical systems 
by the use of transducers, such as chemical synapses, that can decouple two parts of a 
system, similarly to operational amplifiers 1 ' 1 . 

Could neural hardware exploit this model of computation? Increasing evidence shows that 
electrotonic potentials play a primary role in many neurons (Schmitt, Dev and Smith, 1976) 
and that membrane properties such as resistance, capacitance and equivalent inductance 
(arising through voltage and time-dependent conductances; see for instance Cole, 1968 
/""""N and Koch, 1984) may be effectively modulated by various types of neurotransmitters, acting 

over very different time scales (Marder, 1984). Dendrodendritic synapses and electrical 
gap junctions serve to mediate graded, analog interactions between neurons and do 



not rely on all-or-none spikes (Graubard and Calvin, 1976). Such an electrical mediated 
interaction between cells has been exploited in a model of the small-signal behavior of the 
photoreceptor network in the vertebrate retina (Torre and Owen, 1983). 

When implementing electrical networks in equivalent neuronal hardware, one can draw upon 
a large number of elementary circuit elements 13 (Fig. 3). Patches of neuronal membrane 
or cytoplasm can be treated as resistance and capacitance. Voltage sources can be 
mimicked by synapses on dendritic spines (Koch and Poggio, 1983), whereas synapses 
on large dendrites act as current sources. Chemical synapses could effectively serve to 
decouple different parts of a network. Chemical processes such as the reactions associated 
with postsynaptic effects or with peptides could also be thought as part of a complex 
electrochemical network. Obviously, the analogy cannot be taken too literally. It would be 
very surprising to find the exact neural analog of the circuit of Fig. 3 somewhere in the 
CNS. We are convinced, however, that the style of computation represented by analog 
circuits represents a very useful model for neural computations as well as a challenge for 
future VLSI circuit designs. 




Figure 3: This schematic figure illustrates an hypothetical neuronal implementation of the 
regularization solution to the motion problem. A dendrite, acting both as pre- and post-synaptic 
element has a membrane resistance that can vary with location. It can implement under steady-state 
conditions the circuit 1b. The inputs— corresponding to the measurements v- 1 — are given by synaptic 
mediated current injections, while the output voltages — corresponding to the desired v T — are sampled 
by dendro-dendritic synapses. The membrane resistance can be locally controlled by suitable synaptic 
inputs— corresponding to the curvature of the contour— from additional synapses opening channels 
with a reversal potential close to the resting potential of the dendrite. This scheme can easily 
be extended to the case where the measurements of the perpendicular velocities are not exact, 
by having a similar, second dendrite (see also figure 1c). The interaction between both dendrites 
takes place via two reciprocal chemical synapses. If the corresponding cross-term in equation 
(11) is negative, the chemical synapse must be inverting, presynaptic depolarization leading to an 
hyperpolarization. 



The regularization approach is not limited to early vision. Other sensory modalities and 
motor control can be analyzed in a similar way (Poggio and Torre, 1984). For instance, the 
network corresponding to the variational principle suggested by Hogan for controlling the 
formation of voluntary movements (Hogan, 1984) is topologically related to the electrical 
network underlying one-dimensional edge detection. 

The variational principles that we have considered so far for early vision processes are 



quadratic and therefore lead to linear networks. This is not always to be expected. The 
/""N ill-posed problem of combining several different sources of surface information, for instance, 

may lead to non-quadratic regularization expressions and to corresponding nonlinear 
networks. Though this is a topic that still needs to be explored, nonlinearities may greatly 
expand the rather restricted universe of computations that can be performed in terms of 
quadratic minimization principles (Hopfield, 1984). Again, analog networks may be used 
to solve these minimization problems, that will in general have multiple local minima 
corresponding to the zeros of the mixed potential (Brayton and Moser, 1964; Oster, Perelson 
and Katchalsky, 1973). Schemes similar to annealing (Metropolis, Rosenbluth, Rosenbluth, 
Teller and Teller, 1953; Kirkpatrick, Gelatt and Vecchi, 1983; Hinton and Sejnowski, 1983) 
may be easily implemented by appropriate sources of gaussian noise' 5 . In a chemical 
network "noise" may be introduced in various, simple ways. Needless to say, a number 
of biophysical mechanisms, such as somatic and dendritic spikes, interactions between 
conductance changes, voltage and time-dependent conductances etc., are likely to be used 
by neurons and patches of membrane to perform a variety of nonlinear operations. 
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Footnotes 



[1] In 1-D the problem of edge detection is considered as a problem in numerical 
differentiation of the light intensity f(x). Numerical differentiation can be regularized in the 
following way. The model is ?y,- = f(xi). We want to estimate /'. We chose a regularizing 
functional ||./V|| — f[f"(x)) 2 dx, where /" is the second derivative of /. 
The solution can be found by 
(a) minimizing 



/in 



2 dx (1.1) 



subject to the data y it or by 
(b) minimizing 



£(v<W(*0)' + x| [/"(*)]'& (1-2) 

and then in either case by differentiating / to obtain /'. 



[2] Regularization theory provides techniques to determine the best X (Tikhonov and Arsenin, 
1977; see also Reinsch, 1967). For instance, methods such as the Generalized Cross 
j*"*\ Validation method (Wahba, 1980) may be used to find the optimal X. In the case of edge 

detection X effectively corresponds to the scale of the filter (Poggio et al., 1984). 



[3] The main form of Hamilton's Postulate of least action states that the motion of a 
dissipation-less dynamical system, free from external disturbance, from a configuration at 
time <t to another configuration at time t 2 , is such that the integral of its Lagrangian 
L—T-U is stationary on the path followed. That is 



L 



Ldt = Extremum. 



If one extends Hamilton's Postulate to dissipative systems acting under external forces it 
can be shown that 



I 



[L + W)dt = Extremum 



must hold, where W is the virtual work done by the dissipative elements (e.g. resistors) and 
the sources. 



[4] Kirchhoff's laws are physical restatements of the topological properties of the dynamic 
space. For electrical networks they correspond to conservation of flows (KCL) and 
f m \ uniqueness of potential (KVL). Tellegen's theorem capture the basic constraints provided 

by KCL and KVL. It is one of the most general and powerful results of network theory and 
is independent of any assumption about constitutive relations or stationarity. 



Tellegen's theorem: If U is the vector of branch potentials (a component for each branch) 
/"■% and J is the vector of branch flows, then 

U T J = 0. 

Thus the flow and the potential variables are orthogonal at any instant in time. 



[5] If one assumes the electric circuit to be isothermal, then dividing the heat, Q, dissipated 
in the resistors by the temperature T, we find that Q/T = dS/dt. is also minimum (Oster and 
Desoer, 1971). The theorem of minimum entropy production expresses a kind of variational 
property of linear nonequilibrium systems. When given boundary conditions prevent the 
system from reaching thermodynamic equilibrium (i.e. zero entropy production) the system 
settles down in the state of "least dissipation" (Prigogine, 1967). 



[6] In addition to Hamilton's Postulate, there are two specialized variational principles for 
networks composed entirely of sources and linear or nonlinear dissipative elements. In 
such a network, the actual distribution of the currents will be such as to minimize the total 
content G of the system where G is given by 
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= / vdi. 
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Conversely, the distribution of voltages will be such as to minimize the total co-content / 
of the system where 



/ ia 
Jo 



If the network considered contains only linear resistors and sources, then both of the above 
principles reduce to Maxwell's Stationary Heat Theorem: the distribution of voltages and 
currents will be such as to minimize the total power dissipated as heat (MacFarlane, 1970). 



[7] In a network of linear resistances r k = l/g k and current sources /,- the associated 
quadratic power functional to be minimized is given by 



E^J-Ew- 



where the second sum includes all the current sources. 



[8] Assume a Hilbert space with an inner product <•,•>, which defines a quadratic norm 
||-||. An ill-posed problem can be formulated in terms of norms; i.e. minimize 

\\l>zf + \\\Az-bf ' (8.1) 

where P is any linear operator and b is the data. This is the third regularization method in 
Poggio and Torre (1984). Writing this in terms of inner products, we have 



^ m * % - < Pz, Pz > +\ < Az - b,Az - b > . (8.2) 

This is equal to 

< Pz, Pz> +X < Az, Az> -X < Az, b> -X <b,Az> +X < 6, 6 > (8.3) 

Since the last term is constant, it can be disregarded in the minimization. If the adjoints of 
the operators P and A are denoted by /'* and A* respectively, minimizing this expression 
is equivalent to minimizing 

< z, P*Pz > +X < z,A*Az > -2X < z,A'b > . (8.4) 

Defining a new operator Q by Q = P*P + \A*A, we can formulate the original variational 
problem as the problem of minimizing 

< z, Qz > +X' < z,A*b > (8.5) 

The first term can be identified with the total power dissipated in a linear resistive network, 
while the second term is the voltage- or current-source contribution. Note that Q is 
automatically self-adjoint. If Q is a linear operator satisfying 

m < z,z> < < z,Qz > < M < z,z > 

for all z 6 II, with II being a Hilbert space, and some M,m > 0, the vector z minimizing 
^""V eq.(8.5) is the unique solution of the equation Qz = b (thus the energy inner product 

< z,Qz > is H-elliptic and bounded, see Terzopoulos, 1984). The problem of minimizing 
the quadratic functional on a Hilbert space can be formulated as a Hilbert space minimum 
norm problem (Luenberger, 1969). 



[9] Let the Euler-Lagrange equations associated with a quadratic variational problems be 
Qz = b, where the n-dimensional vector b is a function of the data. These equations can 
be implemented in a planar network containing only linear resistances and voltage sources. 
For every variable z { one mesh is set up (with the associated mesh current /<). The mesh 
consists of a battery /?,■ and a resistance r {j = ?,,• for every non-zero entry in the i-th row 
of Q. Moreover, a very simple auxiliary circuit connects the t-th and the .?'-th mesh via an 
auxiliary resistance R tj . The associated circuit current flows not only through %, but also 
through r y of the t'-th mesh and r j{ of the j-th mesh. However, since this scheme requires 
up to ?i 2 /2 resistances, it seems only feasible if Q is sparse. Fortunately, the operator P used 
in the variational problems of early vision (Poggio and Torre, 1984) has a small and limited 
support, so that the corresponding matrix Q will have very few non-zero entries. Although 
this procedure will always yield an electrical network with linear elements implementing 
Qz = b, its physical realization might require negative resistances (if the corresponding 
term in Q is negative), which need to be mimiked by active elements". Alternatively, one 
can use the signal-flow graph techniques of Mason and Coates for solving linear algebraic 
equations (Kuo, 1967). 



f^ [10] S.UIIman (pers. comm.) has suggested an analog mechanical spring model for the 

computation of the smoothest velocity field. Minimization of the potential energy contained 
in the springs corresponds to minimization of equation (5) directly in velocity space. It 



10 



has proved so far impossible to map this very special model into electrical or chemical 
/"**N networks. 



[11] Physically realizable linear resistances, whether in electrical or in chemical systems, 
must dissipate energy, i.e. they are constrained to the upper right and the lower left 
quadrant in the I-V plane and can thus only be positive. However, there are at least three 
different options for implementing negative resistances using basic circuit components, (i) 
The positive and negative resistances can be replaced in a purely resistive network by 
inductances and capacitances, with impedance iuL and -i/(u>C) respectively. The network 
equations are then formulated in terms of the currents and voltages at the fixed frequency 
w. (ii) The negative resistance can be implemented by the use of operational amplifiers 
or similar active circuit elements. A case in point are reciprocal inhibitory synapses, like 
those found between mitral and granule cells in the mammalian olfactory bulb. Depolarizing 
one system leads to a hyperpolarization in the second system, (iii) One can try to exploit 
the negative impedance regions in such highly nonlinear systems as the tunnel diode or a 
Hodgkin-Huxley like membrane. 



f^: 






b) 




Figure 4: Electrical circuits implementing the regularization solution to edge detection (Poggio 
et a/., 1984; Torre and Poggio, 1984) X0 + / = y- Sampling / at discrete points i, wo have 
fi+2 - 4/,+i + G/< - Afi-i + f i+ 2 + X"'A = \~ l yi. Fig. 4a shows the topology of the circuit while fig. 
4b shows an actual realization. The corresponding network equation is 

3-V.-+2 + g t VJ+! - (2g + + 2g~ + g)V t + g+Vi-i + g-Vi-z — h, if 7/ is the current being injected at the 
t'-th node. The negative resistances >_ can be implemented using active components, the resistive 
circuit can, however, be replaced by inductances and capacities and the d.c. voltages and currents 
by a.c. voltages and currents of fixed frequency 11 . 

[1 2] A regularization analysis of the edge detection problem leads tp a simple one-dimensional 
network that implements the variational equation (1.1 j.tHe associated Euler-Lagrange 
equation is the one-dimensionai biharmonic equation — 0, where / is known at some 
discrete points, i.e. /,• = y u where y\ is the light intensity. "The corresponding electrical 
network interpolates f between the sampling points. When the'' measurements are not 
^ exact, the Euler-Lagrange equation is given by X^f + £),. f[xi)s(x - a;,-) == E t - ?/«% - ^i)- 

/"""N The corresponding linear,' electrical Circuit ffrgV'4) approximates jr. Since the corresponding 

elecrical circuits are in both | case? sbace-ihyariaW, he uriderlyincj Ifilters ' are convolution 
filters (Torre and Poggio, T§84; Poggio ct a/., \q§P). '" 
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[13] Two possible chemical and neuronal implementations of the smoothest velocity field 
computation in the case of non-exact measurements are shown in fig. 5. They correspond 
to the electrical circuit shown in fig. 1c. 
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Figure 5: A chemical and a neuronal implementation of equation (11). The cross-term a, which 
can be negative, is mimicked by either an appropriate non-linear chemical reaction between the two 
substances A t and fl,- or by reciprocal synapses. For the details compare with fig. 2a and 3. 



[14] Synapses may implement a variety of linear and nonlinear 1- and 2-ports. A chemical 
synapse converts a presynaptic voltage V pre into a postsynaptic conductance change 
g post (via a chemical process) and corresponds to a non-reciprocal 2-port in the sense 
of Brayton and Moser (1964) or Oster et al. (1971). Its constitutive relation defines a 
2-dimensional surface in the four-dimensional space spanned by V pre ,I pre ,V po , t and l vost . 
The two presynaptic variables, V pre and l pre are related by Ohm's law and similarly for the 
postsynaptic variables. The relation between V pre and V poKt is given by f[x), where / is 
in general a nonlinear function. A chemical synapse is non-reciprocal, since variation of 
any of the two postsynaptic variables does not affect the presynaptic variables. Chemical 
synapses tend to decouple neuronal systems from each other, rather like operational 
amplifiers, since a postsynaptic current can be induced without any change in presynaptic 
current. Depending on /, synapses can mimick positive or negative resistances. Reciprocal 
synapses, like those found between mitral and granule cells in the olfactory bulb or 
between bipolar and amacrine cells in the vertebrate retina (Ellias and Stevens, 1972), 
may reintroduce reciprocity. Because Vi = f(V 2 ) and V 2 = g{V,), where V, and V 2 are the 
intracellular voltages of the two neurons, V t can be varied while observing its effect on 
V 2 . Upon reversing the position of observer and input, one expects to obtain symmetrical 
measurements - at least up to small signals. Electrical synapses, usually referred to as 
gap junctions, provide a low-resistance pathways between cells, thus approximating linear 
resistances. However, rectifying gap junctions are known to exist, implementing the neuronal 
equivalent of diodes. The transjunctional resistance can be varied by a variety of neuronal 
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parameters, such as intracellular calcium concentration, pH and voltage (Bennett, 1972). 



[15] The stochastic differential equations describing an electrical or a chemical system with 
a source of Gaussian noise (for instance voltage or the presence of a chemical reactive 
substance) can be formulated in terms of Ito or Stratonovitch calculus (Wax, 1954; Gihman 
and Skorohod, 1972). In physics the equations are called Langevin equations. They can 
be solved with the Fokker-Planck or the Kolmogorov method. A "solution" of a stochastic 
differential equation is a characterization in terms of probability distributions of the "output" 
process. For linear networks, simpler correlation methods can also be used. If the noise is 
white and Gaussian, its spectral density is proportional to the "temperature" T. The output 
is also a Gaussian process with probability distributions that asymptotically approach the 
Maxwell-Boltzmann distribution. 
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